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ABSTRACT 

This  paper  describes  an  approach  and  method¬ 
ology  for  providing  automated  control  In  a 
multi-process  environment  for  the  display  and 
management  of  tactical  graphical  Icons  In  an 
object-oriented  environment. 


I.  THE  RESEARCH  DOMAIN 


A.  CORPS  MANEUVER  CONTROL  PLANNING 

TheUSArmy  Communications- Electronics  Com¬ 
mand  at  Fort  Monmouth,  New  Jersey,  has  been 
performing  exploratory  research  to  apply  Ar¬ 
tificial  Intelligence  (Al)  technology  to  the  prob¬ 
lem  of  maneuver  control  planning  for  a  corps 
commander.  The  project  consists  of  a  group  of 
coordinated  research  efforts  In  object-ori¬ 
ented  tactical  graphics,  man-machine  inter¬ 
face,  terrain  reasoning,  planning,  plan  recogni¬ 
tion,  knowledge  acquisition,  and  representa¬ 
tion. 

B.  THE  DEVELOPMENT  ENVIRONMENT 

An  experimental  test-bed  was  constructed 
which  consists  of  a  network  of  Lisp  machines 
and  a  large-screen  tactical  dlaplay.  This  pro¬ 
vides  a  state-of-the-art  Al  environment  In 
which  the  capabilities  of  an  object-oriented 
approach  can  be  explored  for  tactical  decision 
aids.  An  icon  on  the  screen  represents  a  Lisp 
object,  and  associated  with  it  can  be  its  graphi¬ 
cal  and  reasoning  attributes,  as  well  as  its 
functionality,  via  message  passing. 

C.  THE  MAN-MACHINE  INTERFACE 

To  the  user,  the  prototype  system  Is  an  Intel¬ 
ligent  plan  editor.  It  monitors  his  Inputs  during 
plan  development  and  provides  critiques.  It's 
designed  to  support  his  planning,  not  to  do  the 
planning  for  him. 

The  prototype’s  man-machine  Interface  pro¬ 
vides  the  following  functionality: 

-  It  brings  system  planning  capabilities  to  the 
user. 

-  It  shows  the  state  of  the  planning  system  and 


database  to  the  user. 

-  It  allows  the  user  to  provide  textual  and  gra¬ 
phical  input. 

-  It  permits  the  user  to  asynchronously  modify 
the  situation,  goals,  and  resources  present  in 
the  various  knowledge  bases. 

•  It  presents  a  computer  mediated  planning  en¬ 
vironment  as  close  as  possible  to  that  in  which 
current  planning  activities  are  carried  out. 

Additional  interface  functionality,  not  yet  im¬ 
plemented,  can  allow  the  user  to  control  the 
display  of  information  and  graphics  on  the 
tactical  displays. 

Currently,  the  prototype  uses  two  display 
monitors.  A  monochrome  screen  displays  a 
command  menu  and  four  plan-editing  windows 
for  textual  input.  Each  window  is  of  a  type  that 
matches  a  particular  planning  function.  The  user 
may  use  the  command  menu  to  select  a  particu¬ 
lar  type  of  window  for  display.  The  second 
monitor  Is  a  color  graphical  display  of  the 
battlefield  background,  overlaid  with  symbol- 
ogy. 

D.  THE  PROCESS  MODEL 

On  a  machine  reasoning  level,  the  maneuver 
controlplanningproblem  was  seen  to  be  best  ex¬ 
pressed  in  terms  of  a  collection  of  asynchro¬ 
nous,  cooperative  processes.  The  user  himself 
Is  considered  a  process.  These  processes  per¬ 
form  different  planning  tasks  and  communicate 
with  each  other  directly  through  message  pass¬ 
ing  and  Indirectly  through  one  or  more  shared 
knowledge  bases.  They  work  in  parallel,  just  'll 
like  the  corps  command  staff.  The  display  x 
windows  on  the  monochrome  display  are  asso-^ 
elated  with  unique  reasoning  processes  and 
provide  the  user  interface  to  them. 

For  the  reasoning  subsystem,  user  control  is  ~ 
causal.  Reasoning  is  data  driven  by  modifica¬ 
tions  to  the  tactical  database.  Plans  are  evalu-  _ 
ated  as  new  information  arrives  or  old  informa¬ 
tion  changes,  and  other  processes  are  invoked  — 
or  spawned  to  evaluate  plan  consistency.  For 
the  textual  and  graphic  displays,  the  user  — 
shares  control  with  the  reasoning  processes. 


E.  PROCESS  COOPERATION  AND  SYMBOLOGY 
CONTROL 

“Screen  clutter  Is  a  major  concern”  [1].  “For 
tactical  applications,  the  transition  to  ADP  sys¬ 
tems  depends,  in  part,  on  a  viable  resolution  to 
the  clutter  problem”  [2].  On  the  textual 
display,  declutter  Is  no  Issue,  as  there  are 
always  four  windows  visible.  The  only  concern  is 
that  of  contention.  When  It  occurs  from  conflict¬ 
ing  requests  by  reasoning  processes,  theuserls 
notified  and  decides.  This  was  not  viewed  as 
being  a  distraction,  as  It  relates  to  the  reason¬ 
ing,  Itself,  and  may  provide  valuable  Insight  to 
the  user  about  how  the  system  is  processing  or 
viewing  the  problem  at  hand.  However,  for  the 
tactical  display,  screen  content  needsto  be  kept 
at  a  minimum.  When  a  process  no  longer  requires 
a  symbol  to  be  seen  on  the  screen.  It  needs  to 
issue  a  request  for  Its  erasure.  This  can  create 
a  conflict  if  another  process  may  also  desire  Its 
display.  To  ask  the  user  to  resolve  matters  as 
they  come  up  on  an  icon  by  icon  basis  Is  distract¬ 


ing.  A  method  of  providing  display  control  In  an 
automated  manner  was  required  and  is  the  sub¬ 
ject  of  this  paper. 

II.  DISPLAY  ACCESS  LANGUAGE 

A.  REQUIREMENTS 

To  provide  the  prototype  developers  a  uniform 
way  of  performing  graphical  operations  and  to 
resolve  the  display  control  Issue  in  an  auto  mated 
manner,  a  display  access  language  was  designed 
and  Implemented. 

1.  GRAPHICAL  REQUIREMENTS  In  conventional 
tactical  command  centers,  tactical  icons  and 
symbology  are  taped  onto  one  or  more  plastic 
overlays  that  may  be  mounted  or  saved.  A 
mechanism  for  grouping  Icons  for  display  ope  ra¬ 
tions  was  therefore  needed.  An  icon  may  be 
placed  on  a  plastic  overlay  that  Is  not  yet 
mounted  on  the  map  and  is  therefore  not  yet 
visible  to  the  user.  Conditional  icon  display  was 
therefore  also  needed.  For  efficiency,  calls  for 


Icon  Attributes 


Name 

Owning-process 

Location 


Assoclated-with- 

overlays 

Visible-on-maps 


Visibility-reasons 


Hlghllghted-on 

maps 


Highlight-reasons 


Type  and  Contents 

Name-of-process 


Point,  Point-list,  List-of-polnt-lists, 
List-of-list-of- point- lists 


Llst-of-names-of-overlays 


((Name-of-map  color-list  t-or-nil)  ...) 


((Name-of-map 

(Name-of-lcon-or-overlay 

name-of-process)...)...) 


((Name-of-map  color-list  t-or-nil)  ...) 


Purpose 

Which  process  owns/created 
the  Icon? 

Where  Is  the  Icon?  How  Is  It 
drawn? 


With  which  overlays  is  the 
icon  associated  with? 

On  which  map  Is  Icon  now 
visible?  Which  color  was 
used?  Did  the  user  request 
icon  declutter? 

For  each  map  that  Icon  is 
visible  on,  was  the  display 
request  for  icon  display  or 
for  overlay  display?  Which 
process  made  the  request? 

On  which  map  is  icon  now 
highlighted?  Which  color  was 
used?  Did  the  user  request 
icon  declutter? 


((Name-of-map 

(Name-ot-icon-or-overlay 

name-of-process)...)...) 


For  each  map  that  Icon  Is 
highlighted  on,  was  the 
display  request  tor  icon 
highlighting  or  for  overlay 
highlighting?  Which  process 
made  the  request? 

Table  1:  Selected  Icon  Attributes,  Attribute  Types,  and  Purposes 
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Overlay  Attributes 

Name 

Type  and  Contents 

Purpose 

Owning-process 

Name-of-process 

Which  process  owns/created  the  overlay? 

Overlay-plane 

Name-of-plane 

On  which  plane  is  overlay?  Which  color? 

On-maps 

List-of-names-of-maps 

On  which  map  is  overlay  currently  mounted  on? 

Overlay-components 

Llst-of-names-of- Icons 

Which  icons  are  associated  with  this  overlay? 

Table  2: 

Selected  Overlay  Attributes,  Attribute  Types,  and  Purposes 

display  operations  needed  to  be  minimized.  The 
system  had  to  know  not  to  Issue  a  call  for  Icon 
display  if  the  icon  was  already  visible.  Also,  the 
system  should  know  not  to  highlight  an  icon  that 
was  not  visible  on  the  map.  Because  tactical 
commanders  often  simultaneously  refer  to  sev- 
eral  maps  of  different  scales,  multiple  color 
displays  had  to  be  managed.  Finally,  a  method  of 
highlighting  or  displaying  an  Icon  in  a  special 
color  was  required. 


method  of  controlling  the  display  and  erasure  of 
each  icon  was  needed.  A  declutter  In  g  mechanism, 
that  is,  a  means  of  providing  the  user  with 
control  and  override  for  an  icon’s  display  In  an 
automated  environment,  was  also  required. 


To  meet  the  graphical  requirements,  a  virtual 
overlay,  a  Lisp  object,  was  designed  with  at¬ 
tributes,  attribute  values,  and  a  defined  func¬ 
tionality.  Tactical  icons  were  given  an  associ¬ 


ated-wlth-overlays  attribute  where  the  names 
of  all  overlays  that  the  icon  was  ‘on’  could  be 
stored  in  a  list.  Every  member  of  this  list  was 
unique.  Overlay  objects  were  given  a  similar 
overlay-components  attribute,  a  list  of  names  of 
icons.  Thus,  graphical  operations  could  be  per¬ 
formed  on  an  single  icon  and  on  a  group  of  icons. 
The  overlay  had  an  on-maps  attribute,  a  list  of 
names  of  map  displays.  This  signified  whether 
the  overlay  was  ’mounted' on  a  particular  map  or 
not.  Bydefault.acallforan  Icon'sdisplaywhen 
its  associated  overlay  was  not  mounted  on  its 
map  would  not  be  executed,  providing  a  mecha¬ 
nism  for  conditional  display.  Associated  with  the 
Icon  was  a  visible-on-maps  attribute,  a  list  of 
lists.  If  no  process  requested  the  icon's  display, 
the  list  was  nil.  Otherwise,  each  sub-list  con¬ 
sisted  of  the  name  of  a  map  display,  the  name  of 
thecolor(s)that  were  used  to  drawtheicon, and 
the  Lisp  atom  t  or  nil.  The  latter  was  used  to 
designate  whether  the  user  requested  the  icon's 
erasure,  for  declutter.  Every  map  name  was 
unique.  Thus,  the  system  could  easily  determine 
whether  a  call  to  display  an  icon  was  unneces- 


Display: 


Highlight: 


User  Override: 


Grouping: 


Utility: 


Display  Access  Language 


Show-Icon 

Erase-Icon 

Highlight-Icon 

Dehighlight-lcon 

Declutter-Icon 

Restore-Icon 

Associate-lcon-WIth-Overlays 

Dissoclate-lcon-From-Overlays 


Move-Icon 


Show-Over  lay-icons 
Erase-Overl  ay -Icons 

High  light-Over  lay-icons 
Dehighlight-Overlay-lcons 

Declutter-Overlay-lcons 

Restore-Overlay-Icons 

Associate- Overlay-With-lcons 
Dissoclate-Overlay-From-lcons 
Clear- Overlay-From-lcons 

Mount-Overlay-Onto-Maps 

Remove-Overlay-From-Maps 


Table  3:  Disnlay  Access  Language  For  Tactical  Graphics 


1 

sary.  Slnca  this  was  storad  aa  a  list,  multiple 
map  displays  could  ba  easily  managed.  Indication 
of  user  override  was  built  Into  the  attribute’s 
structure.  Data  on  Icon  highlighting  was  simi¬ 
larly  storad  In  the  Icon's  hlghllghted-on-maps 
attribute. 

C.  VISIBILITY  AND  HIGHLIGHT  REASONS 


To  minimize  screen  content  and  to  provide  dis¬ 
play  control  In  an  automated  manner,  for  every 
Icon  that  was  called  for  display  or  highlighting, 
the  reasons  associated  with  this  operation  were 
stored  In  the  Icon’s  visibility-reasons  and  high¬ 
light-reasons  attributes.  The  reasons  specified 
the  map  that  the  Icon  Is  to  be  visible  or  high¬ 
lighted  on,  the  process  that  re  quested  the  ope  ra¬ 
tion,  and  whether  the  request  was  for  the  Icon  to 


Show-Icon 

(Show-Icon  ICON  (&key  (map-alu  nil)  (overlays  In-overlays) 
(conditional-show  t)  (caller  owner))) 


Required  Arguments:  ICON,  unique  Icon  Identifier. 


Optional  Keyword  Arguments: 


Data  Type 


Default 


Map-alu 


Overlays 

Conditional-show 

Caller 

Purpose: 


List  of  two  elements.  First  Is  the 
name  of  a  map  object.  Second  Is  the 
name  of  a  color  object. 

List  of  names  of  overlay  objects. 


T  or  nil 


Nil 


Name  of  a  process. 


Overlays  that  icon  Is 
associated  with 


Name  of  process  that 
owns/created  icon 


Draws  Icon  on  map  window  If  not  alraady  visible  and  there  is  no  indication  of  user 
override.  If  there  is  no  entry  for  map  In  Icon's  vlslble-on-maps  attribute,  function 
adds  one.  Entry  Is  of  the  form  (Name-of-map  color-list  t).  If  there  is  no  entry  for 
map  in  Icon's  visibility-reasons  attribute,  function  adda  one.  Entry  is  of  the  form 
(Name-of-map  visibility-reason).  If  there  Is  a  map  entry  but  no  Icon-related 
visibility  reason  for  the  calling  process,  function  adds  one.  Visibility-reason  is  of  the 
form  (Name-of-lcon  name-of-process) 

Options: 

Specification  of  map  window  and  color: 

Uses  map-alu  argjment,  If  provided.  Otherwise,  determines  map  from  specified  or 
implied  overlays,  and  determines  color  from  icon,  it  color  attribute  Is  non-nil, 
or  from  overlays 

Conditional  drawing  of  Icon: 

If  conditional-show  argument  Is  t,  only  draws  Icon  when  overlay  Is  mounted  on  the  map. 
Specification  of  visibility  reason: 

If  caller  argument  is  non-nil,  new  visibility  reason  to  add  for  the  map  is  of  the 
form  (name-of-lcon  caller). 

Example: 

Given  icon  C  with  null  vlslble-on-maps  and  visibility-reasons  attributes,  whose 
owning  process  Is  plan-process  and  which  is  associated  with  overlay  O.  Given  overlay  O 
that  has  not  yet  been  mounted  on  map  M.  Function  call  (Show-icon  C  ’  cond  it  Iona  l-s  how  nil) 
causes  C  to  be  drawn  on  M,  sets  C's  vlslble-on-maps  attribute  to  (M  color-list  T),  and  sets 
the  visibility-reasons  attribute  to  (M  (C  plan-process)). 


Figure  1:  Show  Icon  Syntax  and  Functionality 


be  dlsplayed/hlghllghted  or  whether  it  was  for 
the  overlay  that  the  Icon  la  associated  with  to  be 
dlsplayed/hlghllghted.  These  attributes  were 
lists  of  lists.  Each  aub-llst  was  for  a  unique  map 
display  that  the  Icon  was  visible/highlighted  on. 
The  sub-lists  were  of  the  form  (Map-name  (Vis- 
reason)  (VI  a- reason)...).  Each  Vis-reason  was  of 
the  form  (Name  Process),  where  name  Is  the 
name  ofeltherthe  Icon  or  an  overlay  and  process 
is  the  name  of  the  process  that  requested  the 
operation.EveryVIs-reasonforaglvenmapwas 
unique.  The  structure  of  the  vis-reason  enabled 
an  icon-related  graphic  operation  to  be  made  and 
recorded  for  more  than  one  process,  and  It 
enabled  more  than  one  overlay-relat  ed  graphics  I 
operation  by  a  single  process  to  be  made  and 
recorded.  Thus,  If  a  given  process  had  more  than 
one  reason  for  an  Icon  to  be  seen  or  highlighted, 
it  could  make  an  overlay  for  that  reason,  asso¬ 
ciate  the  overlay  and  Icon  with  each  other,  and 


mustmakeacopyofthe  overlay  and  perform  the 
graphical  operations  on  its  own  copy.  The  map 
for  declutter  operations  can  also  be  determined 
from  the  Icon  or  It  can  be  specified.  Overlays 
mu st  be  specified  for  icon  grouping  functions  and 
icons  must  be  specified  for  all  overlay  grouping 
functions  except  clear-overlay,  which  uses  all 
of  the  icons  in  the  overlay’s  overlay-compo¬ 
nents  attribute.  The  move-icon  function  modi¬ 
fies  only  the  graphics  display  and  the  icon's  lo¬ 
cation  attribute.  For  the  tactical  icons  in  the 
study,  information  In  the  location  attribute  was 
sufficient  to  redraw  the  Icon.  The  mount  and 
remove  overlay  functions  modified  the 
overlay's  on-maps  attribute  and  called  the  dis¬ 
play  or  erase  functions  for  the  icons  in  its 
overlay-components  attribute. 

F.  SHOW  ICON 


have  the  reasons  recorded  and  utilized  in  future 
graphical  operations.  Detailed  descriptions  of 
selected  icon  and  overlay  attributes  are  pro¬ 
vided  in  Tables  1  and  2. 

D.  CONTROL  OF  ERASURE 

With  above  data  structures,  given  a  request  by 
process  Pto  erase  icon  Cwhich  is  visible  on  map 
M,  if  the  vis-reason  (C  P)  was  a  member  of  the 
sub-list  for  M  in  the  icon’s  visibility-reasons 
attribute,  then  it  was  removed.  If  t!<ere  were  no 
more  vis-reasons  for  M,  then  the  sub-list  for  M 
was  also  removed,  the  sub-list  for  M  in  the 
icon's  visible-on-maps  was  removed,  and  the 
icon  was  then  erased.  A  similar  rule  was  fol¬ 
lowed  for  a  request  to  erase  an  overlay  that  C 
was  in.  Dehighlighting  was  handled  In  the  same 
manner.  Thus,  a  process  could  freely  call  for 
symbology  erasure  and  not  conflict  with  the 
display  needs  of  other  processes. 

e  functions  for  sraphical  operation? 

Table  3  lists  the  graphical  functions  that  were 
specified  and  implemented  for  the  initial  version 
of  the  Display  Access  Language.  For  display,  and 
highlighting,  the  maps,  colors,  overlays,  condi¬ 
tions,  and  calling  processes  can  be  determined 
by  default  from  the  icon's  attributes  or  they  can 
be  explicitly  specified.  However,  the  calling 
process  for  erasure  and  dehlghllghtlng  was 
required  to  be  explicitly  specified,  to  minimize 
accidental  erasure.  For  the  corresponding  over¬ 
lay  functions,  the  celling  process  name  that  Is 
used  when  the  graphicsl  operation  is  performed 
to  the  overlay's  components  is  always  the  owner 
of  the  overlay.  Therefore,  to  provide  erasure 
control,  a  minimal  amount  of  cooperation  was 
expected  from  all  processes  (and  the  developers 
who  define  them)  which  Is  that  they  not  request 
a  graphical  operation  to  be  performed  on  another 
processes'  overlay.  If  a  process  needs  the  icon 
grouping  (overlay)  of  another  process,  then  it 


The  syntax  and  description  of  the  show-icon 
function  Is  provided  in  Figure  1.  The  Lisp  key¬ 
word  syntax  permits  the  user  to  specify  the 
optional  arguments  in  any  order,  in  pairs  of 
keywords  and  argument  values.  An  example  of 
Its  usage,  utilizing  message  passing,  is  provided 
on  the  bottom  of  the  figure. 


III.  CONCLUSION 

The  display  language  provides  a  flexible  mecha¬ 
nism  for  tactical  graphics  control  and  display  in 
a  multi-process  environment.  It  provides  sup¬ 
port  for  graphical  functionality  which  emulates 
graphical  operations  in  a  conventional  tactical 
environment  and  it  provides  a  means  of  extend¬ 
ing  this  functionality  In  a  battlefield  automated 
system. 
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